fix(results): keep raw transcripts valid jsonl - #1632

Summary

Provider runs that emitted human-readable stream logs were writing those bytes directly to transcript-raw.jsonl, so consumers expecting JSONL could fail on Codex CLI and app-server bundles. Raw provider logs now remain parseable: existing JSONL logs are preserved unchanged, and non-JSONL logs are wrapped as line records that keep the original text available for inspection.

This keeps the result index contract honest for transcript sidecars while preserving stdout/stderr and target-execution evidence in the existing sidecars.

Validation

• Rebased cleanly onto origin/main after fix(eval): accept target id in targets files #1631 (6f2110b9); current branch head e9e22fec.
• bun run build
• bun test apps/cli/test/commands/eval/artifact-writer.test.ts
• Local OpenAI-compatible run: examples/features/readme-quickstart/evals/my-eval.eval.yaml --target local-openai, score 100%, bundle .agentv/results/2026-07-04T03-53-57-344Z
• Codex CLI run after fix: /tmp/agentv-av-7yhs/provider-sanity.eval.yaml --target codex-cli-local, score 100%, bundle .agentv/results/2026-07-04T04-01-07-928Z; verified transcript-raw.jsonl parses as JSONL
• Transcript import run against closest Codex fixture: examples/showcase/trace-evaluation/fixtures/imported-codex-transcript.jsonl, score 100%, bundle .agentv/results/2026-07-04T04-01-46-126Z; verified Read/Grep/Edit ordering and transcript summary counts
• App-server runtime path was exercised with codex app-server --stdio; it still fails at the protocol boundary with Failed to deserialize JSONRPCMessage, but the failure bundle now emits valid raw JSONL at .agentv/results/2026-07-04T04-01-40-791Z

Follow-up Provider Checks

Using local endpoint http://127.0.0.1:10531/v1, dummy key, model gpt-5.4-mini, and ignored id-based targets under .agentv/cache/av-7yhs/:

• pi-cli-local-openai: PASS, score/pass rate 100%, bundle .agentv/results/2026-07-04T04-42-29-558Z. transcript-raw.jsonl valid JSONL (1092 lines); normalized transcript preserves user then assistant order; index links transcript, raw transcript, metrics, grading, stdout, stderr, and target-execution sidecars.
• copilot-cli-local-openai: PASS, score/pass rate 100%, bundle .agentv/results/2026-07-04T04-43-11-965Z. transcript-raw.jsonl valid JSONL (9 lines); normalized transcript preserves user then assistant order; index links transcript, raw transcript, metrics, grading, stdout, stderr, and target-execution sidecars.
• pi-sdk-local-openai: blocked/error, bundle .agentv/results/2026-07-04T04-42-48-115Z. Child runner exits before provider metadata with pi-sdk child runner exit: exit code 1; direct import check confirms @earendil-works/pi-coding-agent is not installed/resolvable. transcript-raw.jsonl is valid JSONL (2 lines), preserving user then error assistant order.
• copilot-sdk-local-openai: blocked/error, bundle .agentv/results/2026-07-04T04-43-00-087Z. Child runner exits before provider metadata with copilot-sdk child runner exit: exit code 1; direct import check confirms @github/copilot-sdk is not installed/resolvable. transcript-raw.jsonl is valid JSONL (2 lines), preserving user then error assistant order.

Codex CLI was not rerun after the rebase because #1631 changed target id validation/generation, not Codex CLI provider behavior or the raw transcript fix; the current follow-up targets.yaml validates with id, and existing post-fix Codex CLI evidence remains applicable.

Blockers Not Fixed Here

• Codex SDK live coverage is blocked in this worktree because the child runner cannot resolve optional package @openai/codex-sdk.
• Pi SDK live coverage is blocked because optional package @earendil-works/pi-coding-agent is not installed/resolvable.
• Copilot SDK live coverage is blocked because optional package @github/copilot-sdk is not installed/resolvable.

Related

Related: av-7yhs